Systems and methods for time of flight measurement using a single exposure

ABSTRACT

A sensor array arrangement for a time of flight measurement system is disclosed. The arrangement includes a plurality of pixels and circuitry. The plurality of pixels are configured such that a first plurality of pixels receive a first reference signal and a second plurality of pixels receive a second reference signal. The first and second reference signals are phase shifted with respect to each other. The circuitry calculates depth information by combining information from first and second pixel sensor signals. The first pixel sensor signal is based on the first reference signal. The second pixel sensor signal is based on the second reference signal.

BACKGROUND

Time of Flight systems are systems that resolve differences to objectsusing light. Generally, an object is illuminated with a light source, acamera measures light including light from the object, and themeasurements are buffered and processed to determine distance. Thedistance is determined using the speed of light.

For example, Continuous Wave Time of Flight systems typically use fourmeasurements at different phases, exposures, to determine a distance.Calculating the distance from raw sensor data typically requires storinglarge amounts of light measurements stored as data in a buffer which isdone external of the Time of flight sensor chip. The stored data can berelatively large as it necessarily includes multiple exposures, e.g.exposures corresponding to four different phases. Furthermore, thecalculations to determine the distance using the stored data can becomplex. As a result, complex circuitry and relatively large amounts ofpower are required to determine distance in the time of flight systems.

What is needed are techniques to determine distance using the speed oflight with reduced complexity and power consumption.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a system for time of flight measurementusing a single exposure.

FIG. 2 is a drawing illustrating a depth map produced by a depth mapgenerator.

FIG. 3 is a diagram illustrating a sensor that generates pixel sensorsignals based on varied reference signals.

FIG. 4 is a diagram illustrating an arrangement adjacent pixels togenerate pixel signals based on varied phase reference signals.

FIG. 5 is a graph illustrating distance determination for a combineddepth pixel based on varied sensor signals.

FIG. 6 is a flow diagram illustrating a method of operating a time offlight system.

DETAILED DESCRIPTION

The present invention will now be described with reference to theattached drawing figures, wherein like reference numerals are used torefer to like elements throughout, and wherein the illustratedstructures and devices are not necessarily drawn to scale.

FIG. 1 is a diagram illustrating a system 100 for time of flightmeasurement using a single exposure. The system 100 can be utilized todetect objects and distances for the detected objects. An exposureobtains multiple pixel measurements simultaneously at a single intervalof time. Instead of multiple exposures, a single exposure is used togenerate distance information. As a result, consumption of power, memoryand complexity is reduced.

Time of flight (ToF) measurements generally refer to distancemeasurements determined using the speed of light and image/pixelsensors. The distance is typically calculated per pixel and, oncecalculated, can be used for depth detection, gesture identification,object detection, and the like. The distance per pixel is combined tocreate a depth map that provides a three dimensional image.

Other approaches require multiple sequential exposures, also referred toas copies. Each exposure requires light generated from a light sourcebeing amplitude modulating with a respective phase, the phase beingdifferent for different exposures. For example, one approach requiresfour separate exposures, such as 0°, 90°, 180° and 270°. Information ormeasurements from the four exposures is collected and compared todetermine a depth map. A large amount of information needs to be storedand processed. As a result, relatively large amounts of storage andpower are needed. The stored multiple exposures are then used ingenerating the depth map.

The system 100 includes a sensor 104, an analog to digital converter(ADC) 106, a reference signal generator 108, a depth map generator 110,a light source 124 and a control unit 112. The system 100 determinesdistances to one or more objects in a scene 102. The system 100 may be acontinuous wave Time of Flight system such as a photon modulation device(PMD) based Time of Flight system.

The sensor 104 is an image sensor and includes an array of pixelstypically arranged in rows and columns. The number of pixels, rows andcolumns can vary and are selected based on factors including resolution,intensity, and the like. In one example, these sensor characteristicsare selected based on objects to be detected and expected distances tothe objects.

Smaller objects require a higher resolution for detection. For example,finger detection requires a resolution of <5 mm per pixel at a distanceor range of about 0.5 meters. Medium sized objects, such as handdetection, require a resolution of <20 mm per pixel at a range of about1.5 meters. Larger sized objects, such as a human body, require aresolution of <60 mm per pixel at about 2.5 meters. It is appreciatedthat the above examples are provided for illustrative purposes only andthat variations can occur including other objects, resolutions anddistances for detection. Some examples of suitable resolutions includeVGA—640×400 pixels, CIF—352×288 pixels, QQ-VGA—160×120 pixels, and thelike.

The light source 124 is configured to modulate an amplitude of light andemit the amplitude modulated light towards the scene 102. The amplitudemodulation may be based on a reference signal generated by referencesignal generator 108. The reference signal may be a RF signal, e.g. inthe MHz range, although other modulation frequencies can be used. Theemitted light can include light having varied ranges of wavelength, suchas sunlight and infra-red. Light reflects off of one or more objects inthe scene and returns to the sensor 104.

Each of the pixels of the sensor 104 generates a pixel signal based upona time varying reference signal and received light 114 at a single pointin time or time interval. The time interval, in one example, is acontinuous time in which pixels are exposed and sensitive to light. Thetime varying reference signal used in the pixel to generate the pixelsignal and the time varying reference signal provided to the lightsource 124 have substantially the same frequency. The pixel signalscollectively generate an image or representation based on the receivedlight at the pixels. Additionally, the pixel signals 116 are used todetermine a phase difference between the emitted light from the lightsource 124 and the received light 114. The phase difference, alsoreferred to as φ or depth is then used to determine distance to one ofthe objects.

A variety of suitable devices can be used for the pixels of the sensor104. In one example, each pixel includes a photon mixing device (PMD).This PMD includes readout diodes A and B and modulation gates A and B. Areference signal is applied differentially across the modulation gates Aand B while incoming light is received at a photo gate/diode. Adifferential sensor signal is generated across the readout diodes A andB. The sensor signal from a pixel may be integrated for some time inorder to determine phase measurement information.

In operation, for example, a first pixel is provided with a firstreference signal of the reference signals 122. A neighboring pixel isprovided with a second reference signal of the signals 122 and is offsetfrom the first reference signal by a phase offset. The phase offset canbe expressed in degrees and include, 90, 180, and 270 degrees. The lightsource 124 emits light using the first reference signal and the firstand second pixels receive reflected light, where the light is reflectedfrom an object. The emitted light, in one example, is modulated lightswitched on and off with a selected duty cycle and selected frequencyaccording to the first reference signal. Additionally, the emitted lightis at a suitable wavelength, such as infra-red. The first pixel and theneighboring pixel can be referred to as a combined depth pixel. Thus,the first pixel provides a first pixel signal based on the firstreference signal and received light and the neighboring pixels provideneighboring pixel signals based on the second reference signal havingthe phase offset.

The first reference signal and the second reference signal are appliedat the same time to the first and second pixel, respectively. The firstpixel therefore measures the incoming light at a first phase point andthe second pixel measures the incoming light at a second phase pointdifferent from the first phase point. The first pixel signal and theneighboring pixel signals can therefore be used to determine a distanceto an object from the sensor 104 using time of flight. Rather thandetermining distance information for each pixel by subsequently usingdifferent exposures and each exposure having a phase shift to theprevious exposure, the system 100 determines distance information withonly one exposure from measurements in two neighbor pixels. Since onlyone exposure is used, the amplitude modulation of the emitted light iskept unchanged in phase in order to determine the distance information.

The two neighbor pixels are modulated at the same time with referencesignals having a phase shift with respect to each other and the pixelthen combines the measurement information from the two pixels todetermine a distance information. The phase shift between the first andsecond reference signal is not changed in order to determine thedistance information. Thus, the sensing of different phase points of theincoming light signal which is necessary to determine the distanceinformation is not done for each pixel subsequently in different timeintervals (as in a multi-exposure mode) but the different phase pointsof the incoming light signal are sensed at the same time in neighborpixels by using different reference signals. Since the pixels areneighbor pixels, the phase of the incoming light (which indicates adistance) at the two neighbor pixels should vary not too much and acombined depth information can be obtained by utilizing the sensedsignals of the two neighbor pixels. Thus, a combination of these twophase measurements at the two neighbor pixels allows determining thedistance of the incoming light, although in view of the lateral distancebetween the two pixels with slightly less accuracy as with multipleexposures.

In some embodiments, a first group of pixels may receive the firstreference signal and a second group of pixel may receive the secondreference signal. In some embodiments, the first reference signal, thesecond reference signal and the amplitude modulation signal have a samefrequency. In some embodiments, the first reference signal, the secondreference signal and the amplitude modulation signal may keep theirrespective phase in order to determine 3D information. Thus, only oneexposure is required with no phase shifting of the modulated lightsignal in order to determine 3D information from the PMD device.

Generally, the sensor 104 receives reference signals 122 and providespixel signals 116. The reference signals 122, in one example, include afirst reference signal and a second reference signal, where the secondreference signal has a phase offset from the first signal. The referencesignals 122 are typically modulated and have a selected frequency andduty cycle. The phase offset includes, for example, 90, 180, and 270degrees. The first reference signal is provided to odd numbered columnsand the second reference signal is provided to even numbered columns ofthe array of the sensor 104. As a result, the pixel signals 116 includesignals generated using the first reference signal and the secondreference signal, depending on the column.

The reference signal generator 108 generates the reference signals 122used by the sensor 104 and the light source 124. In one example, thereference signal generator 108 includes a phase locked loop thatgenerates a first reference signal. The generator 108 also includesphase offset component that generates a second reference signal by aselected phase offset, such as 90 degrees.

The analog to digital converter (ADC) 106 includes a plurality of analogto digital converter units. Each unit is configured to convert one ofthe pixel signals 116 into a digital signal. The plurality of converterunits, thus convert the sensor signals 116 into digital sensor signals118. The ADC 106 may include at least one converter unit for each of thesensor signals 116. If multiple converter units are available, multiplepixels can be converted at the same time. If at least two converterunits are available, neighboring pixels can be converted at the sametime, and their output can be combined immediately to calculate onedistance value without buffering. Then the two converter units canconvert the next pair of neighboring pixels. In some embodiments, afirst plurality of pixels such as pixels from even numbered columns mayuse a first analog to digital converter unit and a second plurality ofpixels such as pixels from odd numbered columns may use a second analogto digital converter unit.

It is appreciated that other configurations can be used that generatethe digital sensor signals 118 from the sensor signals 116.

The depth map generator 110 is configured to generate a depth map 120based on the digital sensor signals 118. Generally, the depth mapgenerator 110 uses a first signal and a neighboring signal of thesignals 118 to calculate a depth for an associated pixel. In oneexample, a first signal associated with an odd column and a second,neighboring signal from an adjacent column is used to determine depthfor a combined depth pixel. Depths are determined for pixels of thearray to develop the depth map 120. Generally, a pair of neighboringpixels provides a single depth measurement for the combined pair.

In one example, a first image is obtained using the first referencesignal and associated pixels and sensor signals and a second image isobtained using the second reference signal and associated pixels andsensor signals. The depth information is calculated by the depth mapgenerator by combining pixels of the first image with the same orneighboring pixel of the second image. As will be outlined later, thefirst and second reference signals may be signals used to demodulateincoming light in the associated pixels.

The depth map 120 can represent the distances or depths measured foreach pixel in a number of suitable ways. For example, the depth map 120can show darker for closer or shorter distances. The depth map 120 candetect objects and object distance for the detected objects.

The control unit 112 is configured to control components including thesensor 104, the signal generator 108 and the depth map generator 110.The control unit 112 can include a processor and a memory in order toperform. The control unit 112 can be configured to selectcharacteristics, including phase and amplitude, of the reference signals122. The control unit 112 can be configured to control the operation ofthe array of pixel sensors within the sensor 104. For example, thecontrol unit 112 can select rows and/or columns of the array in order togenerate the sensor signals 116.

The control unit 112 can also be configured to control the depth mapgenerator 110 and facilitate generation of the depth map 120.

The depth map 120 can be regenerated at intervals and/or other points intime and compared with prior depth maps.

Typically, the system 100 is used for low power applications and modesin order to mitigate power consumption. However, it is appreciated thatthe system 100 can be used for other applications and modes. It is alsoappreciated that variations in the components of the system 100 arecontemplated, including additional components and/or omission of showncomponents.

In one variation, the system 100 uses the combined pixels to generatethe depth map 120 in the low power mode. However, in a high resolutionmode, the system 100 uses multiple exposures to obtain a higherresolution depth map. In some embodiments, the system 100 canselectively switch between the mode using combined pixels and the modeusing multiple exposures to obtain a higher resolution. The mode usingcombined pixels may be a low power mode with a lower resolution whencompared to the multiple exposure mode or approach. Furthermore, in thecombined pixel mode, the system 100 may be capable to roughly calculate3D data directly on the sensor chip without the need for data processingexternal to the sensor chip. As will be described later, determiningfrom two phase measurements phase information of the received light(which corresponds to distance information) may require for example onlya simple arctan calculation. Such a calculation may be implemented fullyin hardware directly on the sensor chip using gates and/or other logicavailable in hardware. Thus, the combined pixel mode may allowdetermining 3D data fully on the sensor chip without external calculatordevices.

The system 100 can switch from the lower power mode to the higherresolution mode based on a number of factors or conditions. The lowpower mode is also referred to as a low resolution mode. For example, atouchless gesture control system can use the lower power mode initiallyand switch to the higher resolution mode once activity is detected toenable more precise measurements. Another example is face detection,where the lower power mode is used for shape detection. Once a shape isdetected, the system switches to higher resolution mode to measurefacial features. Other variations and uses are also contemplated for thesystem 100.

FIG. 2 is a drawing illustrating a depth map 200 produced by a depth mapgenerator. The depth map 200 can be provided by the depth map generator110 and/or variations thereof. The depth map 200 is provided as anexample for illustrative purposes only.

The depth map 200 is arranged in an array of rows and columns as shown.Each individual block of the map 200 corresponds to a pixel of thesensor array. The map blocks include and/or represent a distance value.For this example, darker shades indicate a shorter distance or a closerobject.

Objects 202 and 204 are shown. The object 202 is shown with a darkershade and is closer than the object 204. A component, such as a controlunit, depth map generator and the like can analyze the map and detectthe objects.

FIG. 3 is a diagram illustrating a sensor 300 that generates pixelsensor signals based on varied reference signals. The sensor 300includes an array of pixels arranged in rows and columns. Each pixelgenerates a signal based on a reference signal and received light at thepixel.

In this example, the array includes N columns of pixels. Each columnincludes the same number of pixels, arranged in the rows. Odd numbercolumns, such as C1, C3, and the like are provided with a firstreference signal having a phase offset of 0 degrees. Even numberedcolumns, such as C2, C4, CN and the like are provided with a secondreference signal having a phase offset of 90 degrees from the firstreference signal. As a result, pixels in the odd numbered columnsgenerate pixel signals based on the first reference signal. The pixelsin the even numbered columns generate pixel signals based on the secondreference signal.

Columns are addressed or selected using a column select component 304.Rows are addressed or selected using a row select component 306. Thecolumns and rows can be selected in order to provide one of the firstand second reference signals and/or to read pixel outputs based on thereceived light. The pixel outputs correspond to the pixel sensor signalsgenerated. Pixels signals from a first pixel and pixel from a neighbourpixel may be provided to different analog to digital converter unitswhere the pixel signals may be converted to digital informationsubstantially at the same time. Thus in some embodiments, pixels ofcolumns with even numbers may be associated with a first analog todigital converter unit and pixels of columns with odd numbers may beassociated with a first analog to digital converter. With the digitalinformation of both pixels being available at the same time, the signalsmay be directly transferred to calculate the distance information whichcan be realized for example fully in hardware on the sensor chip. Thesystem does not need extensive buffering in order to calculate thedistance information as will be explained below.

In other embodiments, three analog to digital converter units may beprovided, the first one being associated with a first column, the secondone being associated with a neighbor column to the left and the thirdone being associated with a neighbor column to the right. Then, a pixelsignal of the first column can be combined with a pixel signal of theleft neighbor pixel to obtain first combined pixel depth information andthe same pixel signal of the first column can also be combined with apixel signal of the right neighbor pixel to obtain second combined pixeldepth information. It is to be understood that the pixel density of thethus obtained 3D image can be the same as for the multiple exposuremode. It is further to be noted that the combined pixel mode may providereduced quality images compared to the multiple exposure mode due to theusage of pixel information at different locations with sampling at onlytwo different phase points. However, for certain applications, e.g. lowpower modes or modes requiring a fast calculation of 3D images, suchreduced quality may be fully acceptable. The above described system thusallows a Time of Flight system to provide 3D images fast and with lowpower.

In order to determine distance from a pixel measurement, the pixel and aneighboring pixel from an adjacent column generate pixel signals. Inthis example, a pixel of a first row in C1 and a pixel of the first rowin C2 are adjacent and form a combined depth pixel 302 for purposes ofdistance determination. The pixel of C1 and the pixel of C2 are used togenerate pixel signals based on the first and second reference signals,respectively. It is noted that a light source, such as the light source124, generates light with an amplitude modulation being based on one ofthe reference signals and having substantially the same frequency as thereference signals. For example, the light source 124 may modulate anamplitude of the emitted light in correspondence with the firstreference signal, which is then reflected and received by the sensor300.

FIG. 4 is a diagram illustrating an arrangement 400 of neighboring oradjacent pixels used to generate pixel signals based on varied phasereference signals. The arrangement 400 is provided to facilitateunderstanding and only shows the adjacent pixels in simplified format.It is appreciated that other pixels or pixel sensors can be utilized.Reference is made to the sensor 302.

The arrangement includes a first pixel 402 and a second pixel 404. Thefirst pixel 402 and the second pixel 404 are configured in this exampleas the combined depth pixel 302 of FIG. 3. The first pixel 402 is ofcolumn C1 and the second pixel 404 is of column C2 and they are in thesame row.

The first pixel 402 is configured to receive light 114 and generate afirst sensor signal 410 based on the received light 114 and a firstreference signal 406. The first reference signal 406 is at a phase of 0degrees, in this example.

The second pixel 404 is configured to receive light 114 and generate asecond sensor signal 412 based on the received light 114 and the secondreference signal 408. The received light 114 is substantially the sameas the light received at the first pixel 402 due to their closeproximity. The second reference signal 408 is at a phase offset of 90degrees compared to the first reference signal 406. In one example, thereceived light 114 is substantially from a light source utilizing one ofthe first and second reference signals 406 and 408.

The distance for the combined depth pixel is determined based on thefirst sensor signal 410 and the second sensor signal 412. Thus, onedistance value is generated per pair of pixels or per combined depthpixel.

FIG. 5 is a graph 500 illustrating distance determination for a combineddepth pixel based on varied sensor signals. The graph 500 is providedfor illustrative purposes and is merely an example. The graph 500 isdiscussed with reference to FIG. 4 and the description of FIG. 4provided above.

The graph includes a first value A₁ using a first reference signal 410and a second value A₂ using a second reference signal 412. The firstreference signal 410 and the second reference signal 412 are offset by90 degrees.

The first value A₁ is at a point intersecting slightly more than zerodegrees and slightly less than 180 degrees. The second value A₂ is at apoint more than zero and less than 360 degrees. Lines from theintersection points are drawn to identify the amplitude A and the angleφ at 502.

Equations which provide these values include, for example:φ=arctan(A ₁ /A ₂),where φ is the phase depth,

where A₁ is the first sensor signal 410 and A₂ is the second sensorsignal 412A=√{square root over ((A ₂)²+(A ₁)²)},where A is the Amplitude

The above equations are provided as suitable examples. It is appreciatedthat other suitable techniques can be used to determine the phase depth.

FIG. 6 is a flow diagram illustrating a method 600 of operating a timeof flight system. The method 600 generates a depth map using only asingle exposure. Additionally, the method 600 uses less power than othersystems using multiple exposures to generate a depth map.

A light source generates emitted light using a first reference signal atblock 602. The light source is configured to generate the emitted lighthaving selected wavelengths and intensity or power. The wavelengths andintensity can be selected according to expected objects and/or expectedobject distances.

A sensor receives reflected light at block 604. The sensor includes aplurality of pixels, typically arranged in an array of rows and columns.Each of the pixels receives at least a portion of the reflected light.The number and arrangement of the pixels can be selected according toexpected objects and/or expected object distances. Smaller and/orfarther objects typically require a greater resolution and, thus morepixels.

In one example, the plurality of pixels are arranged in numbered columnsand numbered rows. The sensor includes column and row select componentsconfigured to select one or more of the plurality of pixels in order toapply reference signals and/or read/measure outputs of the selectedpixels.

The first reference signal and a second reference signal are generatedat block 606. The second reference signal is generated having a phaseoffset, such as 90 degrees, from the first reference signal. In oneexample, a phase locked loop is configured to generate the firstreference signal and a phase shift component is configured to generatethe second reference signal from the first reference signal with thephase offset by shifting the first reference signal by the phase offset.It is noted that the blocks 602, 604 and 606 occur at about the sametime 614, but are shown in separate blocks to facilitate understanding.

The sensor generates a plurality of pixel sensor signals at block 608.The sensor generates the plurality of pixel sensor signals according tothe received reflected light, the first reference signal and the secondreference signal.

In one example, odd numbered columns use the first reference signal andeven numbered columns of pixels use the second reference signals. Thus,the pixel signals from the odd numbered columns and the event numberedcolumns have a phase difference based on the phase offset of the secondreference signal to the first reference signal. As a result, pixels inthe same row of adjacent columns receive substantially similar light,but use varied reference signals. As a result, their generated pixelsensor signals vary based on the phase offset.

A depth measurement generator determines a plurality of depth distancemeasurements according to the plurality of pixel sensor signals of asingle exposure at block 610. The depth measurement generator identifiesor associates pixel signals with combined depth pixels, which are twoadjacent pixels that receive varied reference signals. The adjacentpixel signals are then used to determine a depth or distancemeasurement. The depth measurement generator determines the measureddistances for remaining plurality of pixels. It is noted that thedetermined plurality of distance measurements is generated with a singleexposure.

The depth measurement generator generates a depth map at block 612 basedon the plurality of distance measurements. The depth map includes orrepresents the plurality of distance measurements at each pixel. Thedepth map can be represented in a variety of suitable formats. In oneexample, darker pixels correspond to closer or shorter distances.

Additionally, the depth map can include identified/detected objects.Multiple depth maps can be analyzed to detect movement of detectedobjects, gestures, and the like. Accordingly, the method 600 and/orportions of the method 600 can be repeated.

While the method is illustrated and described below as a series of actsor events, it will be appreciated that the illustrated ordering of suchacts or events are not to be interpreted in a limiting sense. Forexample, some acts may occur in different orders and/or concurrentlywith other acts or events apart from those illustrated and/or describedherein. In addition, not all illustrated acts may be required toimplement one or more aspects or embodiments of the disclosure herein.Also, one or more of the acts depicted herein may be carried out in oneor more separate acts and/or phases.

It is appreciated that the claimed subject matter may be implemented asa method, apparatus, or article of manufacture using standardprogramming and/or engineering techniques to produce software, firmware,hardware, or any combination thereof to control a computer to implementthe disclosed subject matter (e.g., the systems shown in FIGS. 1, 2,etc., are non-limiting examples of system that may be used to implementthe above methods). The term “article of manufacture” as used herein isintended to encompass a computer program accessible from anycomputer-readable device, carrier, or media. Of course, those skilled inthe art will recognize many modifications may be made to thisconfiguration without departing from the scope or spirit of the claimedsubject matter.

A time of flight system is disclosed and includes a sensor, a referencesignal generator, and a depth map generator. The sensor is configured togenerate a plurality of pixel signals in response to received light. Thesensor is configured to use, in parallel, first and second referencesignals for demodulating the received light in first and second pixels,respectively. The reference signal generator is configured to generatethe first and second reference signals. The second reference signal isat a phase offset from the first reference signal. The depth mapgenerator is configured to generate a depth map based on the pluralityof pixel signals.

In one variation of the system, the system also includes a light sourceconfigured to emit light using one of the first and second referencesignals where at least a portion of the emitted light is reflectedtowards the sensor as the received light.

In another variation, the system also includes a control unit coupled tothe sensor and the reference signal generator, where the control unit isconfigured to select the phase offset and select pixel rows and columnsof the sensor.

A sensor array arrangement for a time of flight measurement system isdisclosed. The arrangement includes a plurality of pixels and circuitry.The plurality of pixels are configured such that a first plurality ofpixels receive a first reference signal and a second plurality of pixelsreceive a second reference signal. The first and second referencesignals are phase shifted with respect to each other. The circuitrycalculates depth information by combining information from first andsecond pixel sensor signals. The first pixel sensor signal is based onthe first reference signal. The second pixel sensor signal is based onthe second reference signal.

In one variation, the arrangement also includes a depth map generatorconfigured to obtain a first image based on the first reference signal,a second image based on the second reference signal and generate a depthmap based on the first image and the second image.

A method of operating a time of flight system is disclosed. First andsecond reference signals are generated. Each of the first and secondreference signals have a phase relation with respect to a time varyingsignal. Additionally, the second reference signal has a phase offsetfrom the first reference signal. Reflected light is received at asensor. The sensor generates first and second pixel sensor signals,where the first pixel sensor signal is generated based on the reflectedlight and the first reference signal in a first pixel and the secondpixel sensor signal is generated based on the reflected light and thesecond reference signal in a second pixel. Distance information isdetermined by a depth map generator according to the plurality of pixelsensor signals.

In one variation, a light source generates emitted light by modulatinglight in accordance with the time varying signal.

In another variation of the method, a depth map is generated accordingto the plurality of depth distance measurements.

In another variation, a first reference signal and a second referencesignal are generated. The second reference signal is generated with aphase offset from the first reference signal.

In another variation, a first image is generated using the firstreference signal and a second image is generated using the secondreference signal.

In particular regard to the various functions performed by the abovedescribed components or structures (assemblies, devices, circuits,systems, etc.), the terms (including a reference to a “means”) used todescribe such components are intended to correspond, unless otherwiseindicated, to any component or structure which performs the specifiedfunction of the described component (e.g., that is functionallyequivalent), even though not structurally equivalent to the disclosedstructure which performs the function in the herein illustratedexemplary implementations of the invention. In addition, while aparticular feature of the invention may have been disclosed with respectto only one of several implementations, such feature may be combinedwith one or more other features of the other implementations as may bedesired and advantageous for any given or particular application.Furthermore, to the extent that the terms “including”, “includes”,“having”, “has”, “with”, or variants thereof are used in either thedetailed description and the claims, such terms are intended to beinclusive in a manner similar to the term “comprising”.

What is claimed is:
 1. A time of flight system comprising: a lightemitter to emit light during a single exposure, the emitted light havingno phase change during the single exposure; a sensor configured togenerate a first pixel signal and a second pixel signal during thesingle exposure in response to received light and based on a firstreference signal and a second reference signal, wherein the sensor isconfigured to use, in parallel, the first and second reference signalsfor demodulating the received light in first and second pixels,respectively, to generate the first pixel signal and the second pixelsignal, wherein the second reference signal is at a phase offset fromthe first reference signal, wherein the phase offset is not changedduring the single exposure; a reference signal generator configured togenerate the first and second reference signals having the phase offsettherebetween; a first analog to digital converter to convert the firstpixel signal into a first digital signal; a second analog to digitalconverter to convert the second pixel signal into a second digitalsignal parallel to the conversion of the first pixel signal in the firstanalog to digital converter; and a depth map generator configured togenerate a depth map for the single exposure based on a combining of thefirst digital signal and the second digital signal to a single depthmeasurement value.
 2. The system of claim 1, wherein the depth mapgenerator is configured to determine distances based on a phasedifference between an emitted light and the received light.
 3. Thesystem of claim 1, wherein the sensor includes an array of pixelsarranged in columns and rows, wherein a predetermined first plurality ofpixels receive the first reference signal and generate a first pluralityof pixel signals based on the first reference signal; and apredetermined second plurality of pixels receive the second referencesignal and generate a second plurality of pixel signals based on thesecond reference signal; and wherein the depth map for the singleexposure is further based on the first plurality of pixel signals andthe second plurality of pixel signals.
 4. The system of claim 3, furthercomprising an analog to digital converter arrangement configured toconvert in parallel the first plurality of pixel signals and the secondplurality of pixel signals into digital form.
 5. The system of claim 1,wherein the system is configured to calculate depth informationindicating a distance to an object based on the first pixel signal andthe second pixel signal of the sensor.
 6. The system of claim 5, whereinthe depth map generator is configured to use the first pixel signal andthe second pixel signal to determine a phase depth for a combined pixel.7. The system of claim 5, wherein the first and second pixel areneighbor pixels.
 8. The system of claim 1, wherein the phase offsetbetween the first and second reference signals is 90 degrees.
 9. Thesystem of claim 1, wherein the light emitter is configured to emit thelight based on a predetermined phase relation with at least one of thefirst and second reference signals, wherein at least a portion of theemitted light is reflected towards the sensor as the received light. 10.The system of claim 1, wherein the light emitter is controlled tomodulate an amplitude of the light based on a time varying signal, thetime varying signal having a predetermined phase relation to the firstand second reference signals, respectively, wherein the phase relationis kept constant in order to determine distance information.
 11. Thesystem of claim 1, wherein the sensor is further configured to measurethe received light at a first phase point based on the first referencesignal and a second phase point based on the second reference signal togenerate the first and second pixel signals.
 12. A sensor arrayarrangement for a time of flight measurement system, the arrangementcomprising: a light emitter to emit light during a single exposure, theemitted light having no phase change during the single exposure; aplurality of pixels comprising a first plurality of pixels that receivesa first reference signal and a second plurality of pixels that receivesa second reference signal, the first and second reference signals beingphase shifted with respect to each other, where the first plurality ofpixels generate a plurality of first pixel sensor signals based on thefirst reference signal and the second plurality of pixels generate aplurality of second pixel sensor signals based on the second referencesignal, the plurality of the first pixel sensor signals and theplurality of the second pixel sensor signals being generated during thesingle exposure; a first analog to digital converter to convert theplurality of first pixel sensor signals into a plurality of firstdigital signals; a second analog to digital converter to convert theplurality of second pixel sensor signals into a plurality of seconddigital signals in parallel to the conversion of the plurality of firstpixel sensor signals into the plurality of first digital signals; andcircuitry configured to calculate depth information by combininginformation from the plurality of first digital signals and theplurality of second digital signals.
 13. The arrangement of claim 12,wherein a first pixel of a first column of the plurality of pixels and asecond pixel of a second, adjacent column of the plurality of pixels areselected and configured as a combined depth pixel to measure at firstand second phase points.
 14. The arrangement of claim 13, furthercomprising a depth map generator configured to obtain a first imagebased on the first reference signal, a second image based on the secondreference signal and to generate a depth map based on the first imageand the second image.
 15. The arrangement of claim 12, wherein the firstand second pixels are configured to receive emitted light, wherein theemitted light is based on the first reference signal.
 16. A method ofoperating a time of flight system comprising: emitting light during asingle exposure, the emitted light having no phase change during thesingle exposure; generating a first reference signal and a secondreference signal by a reference signal generator, each of the first andsecond reference signals having a predetermined phase relation withrespect to a time varying signal, wherein the second reference signalhas a phase offset from the first reference signal; receiving reflectedlight at a sensor; generating, in parallel, first and second pixelsensor signals during the single exposure, wherein the first pixelsensor signal is generated based on the reflected light and the firstreference signal in a first pixel and the second pixel sensor signal isgenerated based on the reflected light and the second reference signalin a second pixel; and converting the first pixel sensor signal into afirst digital signal; converting the second pixel sensor signal into asecond digital signal parallel to the conversion of the first pixelsensor signal to the first digital signal; and determining distanceinformation by a depth map generator according to the first and seconddigital signals.
 17. The method of claim 16, further comprisinggenerating the distance information according to an arctan of the firstdigital signal divided by the second digital signal.
 18. The method ofclaim 16, wherein a phase of the time varying signal is unchanged inorder to determine the distance information.
 19. The method of claim 18,where the phase offset is about 90 degrees.
 20. The method of claim 18,further switching to a multi exposure mode, the multi exposure modecomprising: generating the emitted light from the light source bymodulating the light in accordance with a first time varying signalduring a first time interval and in accordance with a second timevarying signal during a second time interval; generating a firstexposure pixel sensor signal during the first time interval and a secondexposure pixel sensor signal during the second time interval; andgenerating distance information based on the first and second exposurepixel sensor signals.